Adapting Query Optimization Techniques for Efficient Intrusion Alert Correlation

Intrusion alert correlation is the process to identify high-level attack scenarios by reasoning about low-level alerts raised by intrusion detection systems (IDS). The efficiency of intrusion alert correlation is critical in enabling interactive analysis of intrusion alerts as well as prompt responses to attacks. This paper presents an experimental study aimed at adapting main memory index structures (e.g., T Trees, Linear Hashing) and database query optimization techniques (e.g., nested loop join, sort join) for efficient correlation of intensive alerts. By taking advantage of the characteristics of the alert correlation process, this paper presents three techniques named hyper-alert container, two-level index, and sort correlation. This paper then reports a series of experiments designed to evaluate the effectiveness of these techniques. These experiments demonstrate that (1) hyper-alert containers improve the efficiency of order-preserving index structures (e.g., T Trees), with which an insertion operation involves search, (2) two-level index improves the efficiency of all index structures, (3) a two-level index structure combining Chained Bucket Hashing and Linear Hashing is the most efficient for streamed alerts with and without memory constraint, and (4) sort correlation with heap sort algorithm is the most efficient for alert correlation in batch.